Simulations of estimating causal effects using undersmoothed HAL
Seraphina Shi
2023-05-10
Simulation 1
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-5 + W + 2.25A -0.5WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.859368 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.684002 1st Qu.:0.5776 1st Qu.:0.0000
## Median :-0.008120 Median :2.0265 Median :0.0000
## Mean :-0.002091 Mean :2.1038 Mean :0.4681
## 3rd Qu.: 0.680109 3rd Qu.:3.3626 3rd Qu.:1.0000
## Max. : 4.927536 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.432 2.759 2.703 4.084 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.8493 2.2846 2.2925 3.5674 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.366 1.757 1.917 3.121 5.000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.245 1.494 2.531 5.000
n = 200
## The average lambda of CV-HAL: 0.0142 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0044 (= 0.3129 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0118, 0.0115, 0.0115, 0.0116, 0.0118, 0.0121, 0.0123, 0.0124, 0.0124, 0.0124, 0.0124 (= [0.8446, 0.8299, 0.8286, 0.829, 0.8406, 0.8601, 0.8737, 0.8797, 0.8794, 0.8814, 0.8846] * lambda_CV## The average fitting time for CV-HAL: 4.9150 seconds## The average fitting time for globally undersmoothed HAL: 7.3811 seconds## The average fitting time for locally undersmoothed HAL: 31.2348 seconds
n = 500
## The average lambda of CV-HAL: 0.0056 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0017 (= 0.2954 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0041, 0.004, 0.004, 0.004, 0.004, 0.0041, 0.0042, 0.0043, 0.0044, 0.0044, 0.0044 (= [0.7114, 0.6953, 0.6941, 0.6953, 0.6956, 0.7073, 0.7306, 0.7531, 0.7661, 0.7696, 0.7751] * lambda_CV## The average fitting time for CV-HAL: 5.3269 seconds## The average fitting time for globally undersmoothed HAL: 10.3145 seconds## The average fitting time for locally undersmoothed HAL: 57.3556 seconds
n = 1000
## The average lambda of CV-HAL: 0.0029 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0008 (= 0.2740 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0018, 0.0017, 0.0016, 0.0016, 0.0016, 0.0017, 0.0018, 0.0019, 0.0019, 0.0019, 0.002 (= [0.6072, 0.5747, 0.567, 0.56, 0.5592, 0.5821, 0.6146, 0.6403, 0.6668, 0.6747, 0.6785] * lambda_CV## The average fitting time for CV-HAL: 5.8674 seconds## The average fitting time for globally undersmoothed HAL: 11.6748 seconds## The average fitting time for locally undersmoothed HAL: 63.8782 seconds
Simulation 2
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 + 2W + 5sin(A^{1.5}) + 2WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.806043 Min. :0.000 Min. :0.0000
## 1st Qu.:-0.669070 1st Qu.:0.646 1st Qu.:0.0000
## Median : 0.014005 Median :2.049 Median :0.0000
## Mean : 0.007538 Mean :2.141 Mean :0.0721
## 3rd Qu.: 0.683668 3rd Qu.:3.417 3rd Qu.:0.0000
## Max. : 3.968169 Max. :5.000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.443 2.830 2.770 4.270 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.9928 2.2725 2.3149 3.5913 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.367 1.785 1.948 3.171 5.000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.304 1.577 2.625 5.000
n = 200
## The average lambda of CV-HAL: 0.0036 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0016 (= 0.3371 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0036, 0.0035, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0035, 0.0034, 0.0034 (= [1, 0.9924, 0.9842, 0.9853, 0.9863, 0.9869, 0.9887, 0.9887, 0.9897, 0.9824, 0.9841] * lambda_CV## The average fitting time for CV-HAL: 4.9046 seconds## The average fitting time for globally undersmoothed HAL: 6.5215 seconds## The average fitting time for locally undersmoothed HAL: 19.8505 seconds
## [1] "proportion of simulations that cannot compute empirical SD:"## 1.2 1.1 1 0.6952 0.4833 0.336 0.2336 0.1624 0.1129 0.0785 0.0546
## 0.740 0.748 0.752 0.774 0.830 0.844 0.838 0.864 0.898 0.902 0.924
## 0.0379 0.0264 0.0183 0.0127 0.0089 0.0062 0.0043 0.003 0.0021 0.0014 0.001
## 0.938 0.950 0.954 0.944 0.958 0.954 0.960 0.952 0.956 0.962 0.966n = 500
## The average lambda of CV-HAL: 0.0010 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0003 (= 0.2885 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001 (= [0.9994, 0.9984, 0.9953, 0.9977, 0.9965, 0.9947, 0.9965, 0.9977, 0.9984, 0.9984, 0.9977] * lambda_CV## The average fitting time for CV-HAL: 11.8757 seconds## The average fitting time for globally undersmoothed HAL: 19.6041 seconds## The average fitting time for locally undersmoothed HAL: 87.6812 seconds
## [1] "proportion of simulations that cannot compute empirical SD:"## 1.2 1.1 1 0.6952 0.4833 0.336 0.2336 0.1624 0.1129 0.0785 0.0546
## 0.946 0.938 0.940 0.934 0.938 0.948 0.952 0.972 0.978 0.984 0.988
## 0.0379 0.0264 0.0183 0.0127 0.0089 0.0062 0.0043 0.003 0.0021 0.0014 0.001
## 0.994 0.994 1.000 1.000 0.998 1.000 1.000 1.000 1.000 1.000 1.000Simulation 3
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 - 3W + 4A + \mathbf{I}(A>2) * 5sin((0.8A)^2 - 2.6) )]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.934453 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.684020 1st Qu.:0.6134 1st Qu.:0.0000
## Median :-0.002415 Median :2.0031 Median :0.0000
## Mean :-0.008094 Mean :2.1258 Mean :0.4374
## 3rd Qu.: 0.669826 3rd Qu.:3.4060 3rd Qu.:1.0000
## Max. : 3.427572 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.436 2.799 2.742 4.194 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.8693 2.2064 2.2701 3.5336 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.3978 1.7655 1.9515 3.1756 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.210 1.555 2.665 5.000
n = 200
## The average lambda of CV-HAL: 0.0026 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0009 (= 0.2837 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0024, 0.0024, 0.0024, 0.0024, 0.0025, 0.0025, 0.0025, 0.0024, 0.0025, 0.0025, 0.0026 (= [0.8706, 0.8682, 0.8726, 0.8927, 0.9219, 0.9584, 0.9378, 0.9093, 0.9588, 0.9761, 0.9854] * lambda_CV## The average fitting time for CV-HAL: 3.7293 seconds## The average fitting time for globally undersmoothed HAL: 6.0426 seconds## The average fitting time for locally undersmoothed HAL: 24.0037 seconds
n = 500
## The average lambda of CV-HAL: 0.0009 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0003 (= 0.2800 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 8e-04, 8e-04, 8e-04, 8e-04, 8e-04, 9e-04, 8e-04, 8e-04, 9e-04, 9e-04, 9e-04 (= [0.9227, 0.9227, 0.9237, 0.9206, 0.9183, 0.9397, 0.9147, 0.9201, 0.9661, 0.9833, 0.9895] * lambda_CV## The average fitting time for CV-HAL: 6.1899 seconds## The average fitting time for globally undersmoothed HAL: 14.7461 seconds## The average fitting time for locally undersmoothed HAL: 85.6112 seconds
n = 1000
## The average lambda of CV-HAL: 0.0003 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0001 (= 0.2699 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04 (= [0.949, 0.949, 0.949, 0.9451, 0.9413, 0.9233, 0.9045, 0.9306, 0.9681, 0.9648, 0.9764] * lambda_CV## The average fitting time for CV-HAL: 11.0246 seconds## The average fitting time for globally undersmoothed HAL: 23.4035 seconds## The average fitting time for locally undersmoothed HAL: 118.1803 seconds
Simulation 4
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-7 -W_1 +
2W_2 - 0.5W_4 - W_1W_3 + 4A + 0.5AW_2)]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-3.77555 Min. :-3.92810 Min. :-1.474 Min. :-0.8258
## 1st Qu.:-0.65351 1st Qu.:-0.66067 1st Qu.: 4.026 1st Qu.: 0.6719
## Median : 0.03050 Median : 0.01513 Median : 5.029 Median : 1.0119
## Mean : 0.02011 Mean : 0.01854 Mean : 5.033 Mean : 1.0108
## 3rd Qu.: 0.69892 3rd Qu.: 0.69151 3rd Qu.: 6.050 3rd Qu.: 1.3491
## Max. : 3.94299 Max. : 3.89645 Max. :10.896 Max. : 2.9696
## W5 A Y
## Min. :-0.4024 Min. :0.000 Min. :0.0000
## 1st Qu.: 0.7775 1st Qu.:1.605 1st Qu.:0.0000
## Median : 1.0099 Median :2.549 Median :1.0000
## Mean : 1.0076 Mean :2.554 Mean :0.6176
## 3rd Qu.: 1.2421 3rd Qu.:3.511 3rd Qu.:1.0000
## Max. : 2.3122 Max. :5.000 Max. :1.0000
Simulation 5
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 -W_1 +
2W_2 - 0.5W_4 - 0.5W_1W_3 + 4A + \mathbf{I}(A>2) *5sin((0.8A)^2 -
2.6))]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-4.10305 Min. :-3.992762 Min. :-0.723 Min. :-1.0645
## 1st Qu.:-0.66856 1st Qu.:-0.665766 1st Qu.: 3.993 1st Qu.: 0.6574
## Median : 0.01409 Median : 0.004221 Median : 5.007 Median : 0.9974
## Mean : 0.01345 Mean : 0.004542 Mean : 5.008 Mean : 0.9924
## 3rd Qu.: 0.68434 3rd Qu.: 0.676313 3rd Qu.: 6.014 3rd Qu.: 1.3343
## Max. : 3.37515 Max. : 4.074948 Max. :10.222 Max. : 2.8006
## W5 A Y
## Min. :-0.1632 Min. :0.000 Min. :0.000
## 1st Qu.: 0.7565 1st Qu.:1.604 1st Qu.:0.000
## Median : 0.9909 Median :2.561 Median :0.000
## Mean : 0.9929 Mean :2.554 Mean :0.476
## 3rd Qu.: 1.2283 3rd Qu.:3.509 3rd Qu.:1.000
## Max. : 2.2176 Max. :5.000 Max. :1.000
Simulation 6
Data structure: \(O = (W_1, W_2, W_3, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 1
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Structural equations F and endogenous variables:
- \(W_1 \sim Uniform(o,1)\)
- \(W_2 \sim Bernoulli(\mu=0, \sigma^2 = 2^2)\)
- \(W_3 \sim N(W_1, 0.25*exp(2W_1))\)
- \(A \sim Beta(v(W)\mu(W), v(W)[1-\mu(W)])\)
- \(Y \sim Bernoulli(Q_0(A,W))\)
- where:
- \(v(W) = exp(1 + 2W_1expit(W3))\)
- \(\mu(W) = expit(0.03 - 0.8log(1+W_2) + 0.9exp(W_1)W_2 - 0.4arctan(W_3+2)W_2W_1)\)
- \(\bar{Q}_0(A,W) = expit(-2 + 1.5A + 5A^3 - 2.5W_1 + 0.5AW_2 - log(A)W_1W_2 + 0.5A^{3/4}W_1W_3)\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in (0,1])\), the causal dose-response curve
## W1 W2 W3 A
## Min. :0.0000212 Min. :0.0000 Min. :-3.20646 Min. :0.008172
## 1st Qu.:0.2550912 1st Qu.:0.0000 1st Qu.: 0.07964 1st Qu.:0.483313
## Median :0.5016742 Median :1.0000 Median : 0.80953 Median :0.661615
## Mean :0.5026247 Mean :0.6984 Mean : 0.80890 Mean :0.632873
## 3rd Qu.:0.7530768 3rd Qu.:1.0000 3rd Qu.: 1.53093 3rd Qu.:0.807451
## Max. :0.9999523 Max. :1.0000 Max. : 5.10017 Max. :0.999333
## Y
## Min. :0.000
## 1st Qu.:0.000
## Median :0.000
## Mean :0.483
## 3rd Qu.:1.000
## Max. :1.000
n = 200
## The average lambda of CV-HAL: 0.0090 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0023 (= 0.2632 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.006, 0.0058, 0.0057, 0.0057, 0.0056, 0.0056, 0.0056, 0.0054, 0.0053, 0.0051, 0.0049, 0.0044, 0.0043, 0.0043, 0.0042, 0.0045, 0.0045 (= [0.6913, 0.676, 0.6703, 0.666, 0.6611, 0.6609, 0.6546, 0.6404, 0.6239, 0.603, 0.5776, 0.5095, 0.4942, 0.4902, 0.4755, 0.5031, 0.5107] * lambda_CV## The average fitting time for CV-HAL: 40.4222 seconds## The average fitting time for globally undersmoothed HAL: 46.4000 seconds## The average fitting time for locally undersmoothed HAL: 159.3079 seconds
